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Abstract 

Three methods of pre-flight navigational preparation were 
evaluated, to address problems of geographic disorientation. 
Thirty pilots either "actively" flew a rehearsal of an upcoming 
flight, "passively" watched a replay of an active rehearsal, or 
participated in a map study session in order to "learn" a low 
level navigation route and the surrounding region. Following 
training, subjects were then asked to navigate along the 
previously learned route in a high fidelity simulator, which 
depicted environmental aspects of a "real" flight. The results 
indicated that the spatial resource demands of active flight 
control while navigating in the training condition overburdened 
any potential learning benefits of "seeing" the training 
environment. The route and survey knowledge acquired in the 
active training condition were significantly poorer than the 
knowledge acquired in map study and somewhat inferior to that 
acquired by the passive group. All groups had equal difficulty 
applying survey knowledge to an unrehearsed and unexpected 
navigation task. 
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Introduction 

Overview 

Navigation is a critical task that is essential to pilots of 
all types of aircraft. This task is generally performed by 
pilots quite successfully. However, pilots occasionally make 
navigation errors and these errors often result in a phenomenon 
called geographical disorientation. This occurs when accurate 
awareness of one's location in the world is no longer available 
to the pilot. This state may simply refer to when the pilot gets 
lost, whether the pilot is aware or unaware of this condition. 
Battiste (1993) defines geographic orientation as awareness of 
one's relationship to physical surface features of a region, 
awareness of one's relative location within that region, and 
one's temporal awareness of when one should be at a particular 
location within that region. 

Williams, Tham, and Wickens (1992) recently completed a 
review of NASA's Aviation Safety Reporting System (ASRS) data on 
geographical disorientation and concluded that geographical 
disorientation is most prevalent among general aviation pilots, 
but that it occurs at all levels of flying experience. Williams 
et al. found that geographical disorientation occurs in both good 
and poor weather conditions. Leading causes were poor cockpit 
resource management (CRM) which led to inattention of awareness, 
bad weather, and poor decision making. 

Maintaining geographical orientation is extremely critical 
in aviation settings. Loss of orientation can possibly lead to 
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severe and fatal consequences such as fuel exhaustion, collision 
with the terrain and/or man-made structures, mid-air collisions 
due to entering congested airspace, or approaching/landing at an 
unintended runway. Several researchers have identified this type 
of disorientation as a key component of a pilot's loss of 
situational awareness (Battiste, 1993; Endsley, 1993; Harwood, 
Barnett, & Wickens, 1988; Sarter & Woods, 1991; Wickens, 1992). 

While various methods of navigation exist, pilotage is a 
form of visual navigation that is commonly used by pilots and it 
will be explored in this paper. Pilotage involves visually 
locating a series of landmarks while flying at relatively low 
altitudes. A flight path is typically plotted out on an 
aeronautical chart before a flight is performed. While airborne, 
a pilot determines his/her route and position by comparison of 
map features to their corresponding visual counterparts in the 
outside world. Efficient use of pilotage includes a flight path 
which has landmarks that can be easily identified such as 
mountains, bodies of water, cities, and large man-made objects 
(Cessna, 1986) . 

An example of how pilotage is performed can be illustrated 
in Figure 1. Here the pilot would fly along the depicted course 
by making judgements of distance and orientation between 
landmarks (such as the mountain peak, the bend in the river, and 
the lake) and specify the relationship of these landmarks to 
their corresponding depictions on an aeronautical chart or map. 
The pilot needs to continuously use a process which confirms the 
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location of outside visual landmarks relative to their location 
on a map in order to achieve and maintain geographic awareness. 

If the pilot cannot verify this relationship, then geographic 
awareness is lost. 

Because helicopter pilots often fly low to the ground, 
pilotage is a critical component of their navigation task. The 
huge demands of visual navigation in helicopter flight were 
demonstrated in a study by Sanders, Simmons, and Hoffman (1979). 
They examined the visual workload of U.S. Army copilots/ 
navigators during nap-of-the-earth (NOE) terrain flight and found 
that over 92% of the copilots' total visual time was devoted 
solely to navigational duties. These results reflected the 
tremendous importance of geographical orientation and its 
maintenance, especially at low altitudes. In response to these 
results, a study by Cote, Krueger, and Simmons (1983) was 
performed, which tested if new automatic navigation equipment 
reduced geographic disorientation and high workload problems of 
NOE flight. The results of this study indicated that helicopter 
crews who only navigated with a hand held map made significantly 
more navigational errors than did crews using the map in 
conjunction with the new navigational equipment. 

Geographic awareness needed for this type of low altitude 
navigation is usually assisted through some combination of map 
study and previous experience in an environment. For many years, 
map study was the only method of becoming familiar with the 
navigational aspects of an upcoming flight. However, recent 
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technology has made available the possibility of participating in 
a "rehearsal flight" scenario (e.g.. Bird, 1993; Williams & 
Wickens, 1993). A rehearsal flight is a relatively new method 
which uses a simulator, replicating every aspect of a future 
flight, so that a pilot can practice flying and navigating within 
the unfamiliar environment. This form of practice or training 
could possibly promote greater navigational performance on a real 
flight relative to a condition in which such training had not 
taken place. This rationale is based on the fact that the pilot 
can benefit from "seeing" aspects of an unfamiliar region which 
would otherwise only be available from a map representation. A 
strong argument can be made to develop such technology in if fact 
it proves to be an effective aid to promoting geographic 
knowledge and awareness. 

There are, however, both theoretical and practical 
implications surrounding the potential adoption of this 
technology, relating to how geographic information is stored, 
processed, and operated upon by the pilot. For example, how 
"realistic" should a rehearsal flight environment be? Should 
pilots actively explore an environment to be learned, or can they 
be passively taken through it while watching a video? While 
exploring a new environment, should a map be held in a particular 
manner? Which of the many types of navigational tasks are best 
learned by rehearsal flights? These questions, partially 
addressed in an earlier investigation by Williams and Wickens 
(1993), will be the focus of the current study. 
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In the followii. f cages, many different studies are reviewed 
in order to develop the background of particular topics which 
relate, to varying extent, to the issues addressed in the present 
experiment. The review in particular covers studies that have 
considered the nature of geographic knowledge and how this is 
influenced by the learning environment in which that knowledge is 
acquired. Much of it does not have a direct bearing on rehearsal 
flight technology, but since the effectiveness of that technology 
clearly depends upon the nature of geographic knowledge, its 
partial relevance is evident. In particular, we discuss two key 
issues; differences in the frames of reference between map 
knowledge and the need to use that knowledge in navigation, and 
the nature of geographical knowledge itself. 

Frames of Reference 

A critical component to specifying the relationship between 
visual landmarks and their counterparts, on either a physical map 
or a learned map stored in memory, is matching sources of 
information which have two different frames of reference. 

Several researchers have described two frames of reference used 
in navigation (Aretz, 1991; Battiste, 1993; Harwood & Wickens, 
1991) and these are referred to as an ego-centered reference 
frame (ERF) and a world-centered reference frame (WRF). ERF's 
determine how a pilot orients himself relative to the surrounding 
environment and this reference frame is defined relative to a 
primary axis, the pilot's forward field of view. Landmarks or 
objects are defined to exist to the left/right, front/back, and 
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above/below relative to the pilot. WRF's are usually established 
with a predetermined canonical orientation of north-up. 

Therefore, objects in the environment have an absolute, 
independent relationship in the world relative only to 
directional bearing (such as north, south, east, west). 

The contrast between reference frames may be further 
exemplified in Figure 1. Here, the mountain peak is 
approximately 10 miles to the northwest of the lake. This 
relationship is permanent and it is independent of the pilot's 
view. However, the mountain peak may only be characterized as to 
the left (and more specifically at the 11 o'clock position) if 
and only if the pilot is flying along the depicted heading and 
course. The origin of the ERF is the pilot so while the pilot is 
constantly moving and changing, so are the ERF descriptions. 
Therefore matching the changing ERF with the non-changing WRF, to 
verify that the two are congruent, may be difficult. 

Aretz (1991) and Battiste (1993) propose a model of 
navigation in which four cognitive operations are necessary to 
maintain geographical awareness via ERF-WRF comparisons. (1) 
Triangulation is the process where the geometries of the WRF and 
the ERF are established. This is done by comparing the pilot's 
forward field of view (ERF) with a map (WRF) and determining each 
reference's orientation. (2) Mental rotation then aligns these 
two frames of reference. (3) Image comparison then determines 
if in fact both reference frames are aligned. (4) Translation 
then monitors the ERF's position as it proceeds through the WRF. 
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Within the four step process, Aretz also speculates that two 
separate mental rotations are performed. A circular mental 
rotation parallel to the ground surface brings the WRF into a 
track-up alignment. Then a forward mental rotation translates 
the track-up information on the map into the forward field of 
view. This second process may be alternatively described as 
three dimensional "envisioning." 

As described in the Aretz (1991) model, mental rotation 
plays an essential part in the discussion of ego and world- 
centered frames of reference. Sevetal researchers have 
established that the time required for mental rotation is 
directly proportional to the angular disparity between two images 
to be compared. This finding holds true whether the images are 
two dimensional objects (Aretz & Wickens, 1992; Cooper, 1989), 
three dimensional objects (Shepard & Metzler, 1971; Barfield, 
Sandford, & Foley, 1988) or, in the case of navigation, a forward 
field of view with an ERF and a two dimensional WRF map view 
(Aretz, 1991; Evans & Pezdek, 1980; Sholl, 1987). Because time 
is required to carry out mental rotation, it is not surprising 
that judgements of ERF-WRF correspondence are facilitated when 
the map is physically rotated to a track-up orientation (Aretz, 
1991) . Hence the nature of map orientation (fixed versus 
rotated) may be anticipated to influence the quality of knowledge 
stored during a rehearsal flight, an issue we address in the 
current study. 
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Geographical Knowledge 

While a discussion of ERF-WRF comparisons has focused most 
directly on comparing two visual stimuli, pilots can also be 
assisted in the ERF/WRF maintenance if they have accurate 
knowledge of the environment as well as an accurate physical map. 
It is this creation of knowledge that is, of course, the 
objective of rehearsal flights. It is generally accepted that 
the mental representation of this knowledge may consist of 
various types of geographic knowledge (Gluck, 1991). Thorndike 
(1980) proposed that 3 levels or stages of geographic knowledge 
are created as the navigator learns about a geographical 
environment. Landmark knowledge is the first stage of 
information to be acquired. Salient landmarks in the environment 
(such as large buildings, bridges, and lakes) are visually 
processed so that future recognition will aid in orienting 
oneself in a new and unlearned environment. This builds a 
foundation in memory which aids in the learning of Thorndike's 
second stage, route knowledge. 

Route knowledge is used to navigate from one specifically 
known location to another. This knowledge is typically derived 
from direct navigational experience in the environment. A person 
has an internal "route list" which contains a sequential record 
of visually encoded landmarks to be used in a navigation task. 
Additionally, ego-referenced directional information (e.g., 
taking a left on First Street and then the second right after the 
bank) along the route of landmarks must also be known in order to 
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successfully reach the next landmark on the route list. This 
information is usually first encoded by following a specific list 
of navigational or directional commands. A common example is 
following verbal or written directions to a particular, 
unfamiliar location. Thorndike and Hayes-Roth (1982), however, 
suggest that a person's route knowledge involves more than just 
knowing a particular list of landmarks and directions. Knowledge 
of the relative distance between two points, the angle of turns, 
and general terrain features along segments of the route are 
often encoded as well. 

The building of a spatial data set in this fashion xaads to 
the final level of spatial knowledge proposed by Thorndike 
(1980). Survey knowledge is described as a global understanding 
of the location of objects in the environment within a fixed 
world-referenced coordinate system, the interobject Euclidean 
(straight-line) distances, and the specific as well as the 
general features of a region. In essence, it is a culmination of 
many forms of landmark and route knowledge, but the difference is 
that exact positions are known relative to all others. This 
mental representation of spatial information has been described 
as a "cognitive map" (Tolman, 1948) which may be considered the 
mental analog of a traditional, physical map. Ideally, one can 
navigate and orient from this cognitive map equally as well as 
from a physical map. Williams and Wickens (1993) referred to 
this use of survey knowledge as functional survey knowledge . 
However, cognitive maps are sometimes distorted from their 
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physical counterparts. Several studies have concluded that 
systematic distortions of geographical mental representations (or 
cognitive maps) do occur quite frequently (Battiste & Delzell, 
1991; Chase & Chi, 1979; Stevens & Coupe, 1978; Tversky, 1991). 

Thorndike and Hayes-Roth (1982) conducted an experiment 
which tested how the different types of spatial knowledge are 
derived from different sources. Subjects were assigned to either 
a map study or navigation condition. The map study condition 
required learning the floor plan of the Rand Corporation building 
until the subjects were able to draw a map of the floor plan 
without error. The map study subjects did not have any previous 
experience in the building. The navigation subjects acquired 
knowledge of locations within the building solely through direct 
navigational experience (i.e. walking). This group's 
navigational experience in the building was limited to either 1- 
2, 6-12, or 12-24 months. These subjects were not explicitly 
given map study experience, although there was nothing that would 
have prevented them from occasionally viewing a map of the 
building. Later, all subjects were then tested on a variety of 
spatial judgements. These included judgements of route distance. 
Euclidean distance, relative bearing (orientation) from an actual 
position or from an imagined position (simulated orientation), 
and indication of a relative position on an abstract map that 
contains only two reference points. 

Generally, the investigators found that the type of learning 
(map study versus navigation) differentially influenced survey 
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versus route knowledge learning early in practice, but not later. 
First, the map study group did perform better in judgments of 
Euclidean distance and object location than the low experienced 
navigation group. This was an intuitive finding since map 
learners acquire a "bird's eye" view of the environment and thus 
such global judgments would have been expected to be easier to 
perform. Conversely, the navigation group did better in 
judgments of route distance. This finding is also not surprising 
since the navigation group could mentally simulate past 
navigational experiences through the building while the map study 
group could only make estimates based on the bird's eye view. 

The navigation group also performed better on judgments of 
orientation bearing. Here, the investigators concluded that any 
difficulty in computing bearing by using knowledge of an indirect 
route (presumably learned by the navigation group) was outweighed 
by the more difficult task of mentally rotating a map 
representation (learned by the map study group) in order to 
compute bearing. 

Secondly, the most experienced navigation group performed as 
well as or better than the map study groups on all tasks. This 
indicates that direct navigational experience eventually produces 
a superior cognitive map. Even when map study groups were given 
additional training they did not significantly improve on 
judgments of route knowledge. This lack of improvement suggests 
that there is a limit of spatial knowledge to be obtained from 
map study. Also, navigational experience would seem to be 
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essential if the best results are required. There is a cost with 
navigational experience however. Thorndike and Hayes-Roth (1982) 
found that this process takes a long time to produce the superior 
cognitive map. The subjects with the highest navigational 
experience had between 1 and 2 years where as the map study 
subjects reproduced the map in approximately 20 minutes. 

Evans and Pezdek (1980) found similar results in an 
investigation which map study subjects took longer to make 
correct judgements of orientation than did subjects who only had 
an active navigational experience. These results provided strong 
evidence that active experience in the environment may lead to a 
superior form of survey knowledge than would otherwise be gained 
from map study. The active subjects demonstrated that their 
"functional" survey knowledge may be used to make spatial 
judgments from several different perspectives, without suffering 
in speed or accuracy. 

A study by Hirtle and Hudson (1991) evaluated the route and 
survey knowledge of subjects who had passive exposure to an 
environment or exposure only from map study. Passive group 
subjects were shown a 6 minute slide presentation of a route 
taken through an actual town while map study subjects studied a 
map with a route clearly marked. After the exposure, both the 
passive and map study subjects had demonstrated equivalent, and 
fairly accurate, judgements of route distance. However, the 
passive group had significantly poorer performance on judgements 
of straight line distance and landmark orientation. Hirtle and 
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Hudson concluded that the map study subjects acquired better 
survey knowledge than did the passive subjects while both groups 
developed equivalent forms of route knowledge. The results of 
this study clearly indicate that passive exposure to the 
environment without reference to a map may not provide the 
essential aspects necessary for accurate judgements of spatial 
relations. This is not surprising since the "cognitive geometry" 
needed to perform such judgements (from only a single passive 
route exposure) requires very complex calculations. 

Another study which tested the effects of active and passive 
exposures to the environment was conducted by Gale, Golledge, 
Pellegrino, and Doherty (1990). Subjects between the ages of 9 
and 12 years were exposed to two neighborhoods by either actively 
walking through the neighborhoods or by watching a videotape of 
an active subject's exposure. All subjects were later evaluated 
on scene recognition using photographs, a map drawing test, and 
an actual neighborhood navigation test. The results indicated 
that both active and passive subjects performed equally as well 
on the recognition and map drawing tests. However, the 
navigation test of a particular route revealed that the active 
siabjects, with just a single exposure to the neighborhood, 
performed much better that the passive subjects who received five 
videotape exposures. Gale et al. suggested that these results 
could have been accounted for by two reasons. First, the active 
groups were required to make decisions and then produce the 
necessary motor responses in order to accurately navigate along 
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the desired route while the passive subjects did not have any 
"decisions" to make. Also, the active subjects had a panoramic 
field of view along the route, while the passive subjects were 
restricted to a more limited the forward field of view presented 
by the videotape. These two explanations may have afforded the 
active group with a greater acquisition of route knowledge than 
was otherwise obtained by the passive subjects. 

The studies described above have all involved ground 
navigation. Two studies with a greater degree of direct 
relevance to the current research, because they simulated 
airborne flight, were conducted by Terrell (1990) and Aretz 
(1991) . 

Terrell (1990) compared different forms of navigational 
training received by U.S. Army navigators. The control group 
received the standard, in-flight navigation training normally 
administered while another group received a new video/computer- 
based instruction called Map Interpretation and Terrain Analysis 
Course (MITAC). The MITAC program provided navigators with video 
footage of an actual NOE flight while the navigators tracked the 
flight's progress with a hand held map. The video occasionally 
stopped and identified useful landmarks as well as tested 
navigators judgements of current position within the environment. 
This allowed the navigators to match the video footage (an ERF) 
with the map (a WRF) without the burden of time constraints 
characteristic of an actual NOE flight. Both groups were later 
given an in-flight navigational test in an unpracticed and 
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unfamiliar region. The results indicated that the MITAC groups 
made significantly fewer navigational errors than did the control 
group. Terrell concluded that the video/computer-based training 
(a passive exposure) improved navigational performance over the 
groups who received in-flight navigational training (an active 
exposure). However their research did not fully generalize to 
the rehearsal flight scenario, since the region in which subjects 
were tested, was different from that in which they were trained. 
That is, they were studying the acquisition of general 
navigational skills, rather than specific geographical knowledge. 

Aretz (1991) also contrasted active with passive exposure to 
an environment as subjects actively controlled or passively 
watched a flight simulation through a computer-generated 
environment. Spatial knowledge was assessed by having subjects 
indicate a direction to a landmark, determining if a particular 
forward field of view (ERF) matched a particular position on a 
map (WRF), and by drawing a map of the experienced environment. 
The two groups differed only on the map drawing task. The 
passive groups seemed to produce better maps of the environment 
(measuring survey knowledge) than did the active groups. Aretz 
suggested that the active role of controlling the flight 
simulation (maintaining altitude and heading) may have 
overburdened limited spatial resources needed for navigating, and 
therefore the passive subjects were able to devote all of their 
spatial resources to learning the environment. 

Obviously, a key component distinguishing map study 
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knowledge from active or passive exposure is the reference frame 
by which the geographic information is learned, processed, and 
stored, and we have seen that the ability to integrate geographic 
information based on WRF and ERF's is a crucial aspect of 
geographical awareness. Previously mentioned research in this 
field suggests that there are unique differences between ERF's 
and WRF's (Aretz, 1991). Thorndike and Hayes-Roth (1982) made a 
similar suggestion that people operate off of different reference 
frames depending on how spatial information is originally 
encoded. Therefore, certain navigational tasks may be easier to 
perform depending on which type of reference frame is utilized. 
The results from the Thorndike (1980) and Evans and Pezdek (1980) 
studies indicate that both active and passive exposure to an 
environment (an ERF exposure) provides better survey knowledge 
than does map study. Several other studies (Aretz, 1991; Gale, 
Golledge, Pellegrino, & Doherty, 1990/ Hirtle & Hudson, 1991; 
Terrell, 1990) concluded that passive exposure may be better than 
active exposure for the acquisition of survey knowledge. 

Rehearsal Flight Research 

While the research described above addresses the 
effectiveness of different features of navigational training, in 
the acquisition of different components of navigational knowledge 
and skill, none of these studies explicitly examined the benefits 
of rehearsal flights in transferring to more operational flight 
environments. A review of the literature suggests that this area 
has benefitted from little systematic studying. A preliminary 
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Study by the U.S. Air Force (Martin & Lidderdale, 1983) explored 
the use of rehearsal flight simulations of the F-16 aircraft. 
Experienced fighter pilots were briefed and they prepared for two 
attack plans using standard, non-rehearsal preparation 
techniques. These pilots later flew their planned combat attacks 
using a rehearsal flight system instead of actual aircraft and 
data from these flights indicated that several rehearsal attack 
flights improved the success of these missions. 

Martin (1993) stated that many of the pilot recommendations 
for improving the rehearsal system were used in a more advanced 
rehearsal system, implemented in a study by Nullimeyer, Bruce, 
Conquest, and Reed (1992). There investigators hypothesized that 
the addition of flight rehearsal, to the mission preparation 
process, would enhance the performance and success of an actual 
attack mission. A rehearsal flight trainer, simulating an MH-53 
helicopter, was used in conjunction with the efforts of many 
specialists (such as weather, intelligence, communication, and 
weapons) for mission preparation. Pilots reported that an 
especially useful option used during the mission rehearsal was 
the capability to "freeze" and then "slew" the simulation in 
order to study the attack scenario from many different 
perspectives without the time pressures of flight. Following the 
preparation process, 10 pilots flew their actual helicopters on a 
night attack mission. Post mission debriefing, analysis, and 
pilot reports indicated that the addition of the rehearsal system 
was effective in improving their actual performance relative to 
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past experiences without such rehearsals, thereby testifying to 
the potential benefits of rehearsal flights. However, it is 
important to realize that these conclusions were based upon 
subjective data rather than actual performance measures. 
Furthermore, since there was no control group that flew the 
transfer flight without the benefit of rehearsal flight, it was 
impossible to validate the true value of the latter in developing 
route knowledge for rehearsal flight. While these studies 
demonstrate an interest in rehearsal flight systems, they do not 
clearly establish the effectiveness of rehearsal flights for the 
purposes of promoting route or survey knowledge. 

As we have noted, none of the research on geographical 
representation of knowledge specifically examined a navigation 
flight transfer task. While the U.S. Air Force research did use 
such a task, it did not systematically vary the features of 
training (active/passive/map study) in such a way as to allow 
conclusions to be drawn regarding which of these are most 
important. Only a study by Williams and Wickens (1993) has 
coupled both of these elements, as well as the topics in the 
previously reviewed material, in the context of a rehearsal 
flight scenario. Since this study set the foundation for the 
current experiment, it will be described in some detail. 

In Williams and Wickens (1993), subjects (all current 
pilots) were placed into three environmental exposure groups 
which were classified as either map study, active, or passive 
rehearsal. Map study subjects were given 30 minutes to 
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specifically learn the route and region depicted on the map. The 
other subjects either "actively" controlled a rehearsal flight of 
the depicted route or "passively" watched a replay of an active 
subjects' rehearsal flight. These rehearsal flight groups were 
further differentiated by two levels of scene detail (high or 
low) simulated by the rehearsal flight. After subjects 
participated in learning the route and region, they were 
transferred to a very high fidelity simulator (simulating 
environmental aspects of a real flight) and they were examined on 
their ability to fly the same route, again with use of the same 
map. The map was not constantly available, but it could be 
illuminated at the subjects' discretion. 

Horizontal and vertical tracking errors, recorded during the 
transfer flight, indicated that there was a significant effect of 
the type of initial exposure. Further analysis indicated that 
the active subjects navigated more accurately than did the 
passive subjects, while the map study subjects did not 
significantly differ from either of the rehearsal flight groups. 
Also, the level of scene detail did not appear to influence the 
performance of either the active or passive subjects. 

At the end of the transfer flight, all subjects were 
challenged to fly back to the initial starting point when they 
reached (what they thought would be) the last point on the 
depicted route (a test of functional survey knowledge). Analysis 
of this performance did not indicate any significant difference 
between any of the groups. After the transfer flight, all 
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subjects were then requested to reconstruct the map used during 
the previous sessions. The maps were later scored by two 
measures. A Landmark score was composed of the sum of landmarks 
placed in their correct, relative position and a Mapscore was 
composed the Landmark score divided by the mean bearing error of 
the landmarks' true location. Map study subjects were found to 
have significantly higher Landmark scores compared to the other 
groups. However, the Mapscore measure did not differ 
significantly between any of the groups. Williams and Wickens 
observed that while the map study subjects did well on both of 
the map reconstruction scores (a typical assessment of survey 
knowledge), they were no more successful than the other groups in 
applying this knowledge in order to fly back to the initial 
starting point (a measure of functional survey knowledge). They 
also concluded that the ability to reconstruct the maps (a 
measure of survey knowledge) may be independent of the ability 
the accurately navigate along the route (a measure of route 
knowledge). 

The results of the Williams and Wickens (1993) study were 
both informative and provocative, suggesting that simple map 
study could provide just as effective route knowledge and more 
effective survey knowledge, as the more expensive computer-based 
training methods. The results also suggested that "traditional" 
methods of assessing survey knowledge (map drawing), do not 
necessarily assess the ability of that survey knowledge to solve 
functional navigation problems. The present study was designed 
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to replicate many of the aspects of Williams and Wickens 
research, but also to specifically address two potential 
shortcomings in their methodology. First, in the Williams and 
Wickens study, all subjects had ready assess to the map during 
the transfer flight. This might have "diluted" differences 
between groups, providing the two rehearsal groups with 
additional access to survey knowledge. Second, no control was 
exerted to ensure that the map study group actually devoted the 
same effort to study the route and region, as did the rehearsal 
flight groups. An equivalent period of time (30 minutes) was 
made available for all three groups, but they were not 
specifically instructed to use it all. Finally, Williams and 
Wickens' final measure of functional survey knowledge (accuracy 
of returning to the starting point) did not fully capture all 
aspects of this task. The experiment reported here uses 
essentially the same procedure as the Williams and Wickens study, 
but addresses these three shortcomings. In addition, the time 
allocated for geographical learning (by map study or rehearsal 
flight) was shortened from 30 minutes to 20 minutes, a change 
that has some important implications. 

In the present experiment, subjects (all of whom were 
private pilots) were again divided into three groups and tasked 
to learn a predetermined navigational route, very similar to the 
route used in the previous study. The subjects were exposed to 
the navigational route either through a 20 minute map study 
session, an "active" rehearsal flight simulation with the 
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assistance of a hand held map, or a "passive" viewing of a replay 
of an active subject's rehearsal flight, also with the use of the 
map. Unlike the procedure employed in the Williams and Wickens 
study, active and passive groups were directed to use the map (in 
the rehearsal flight) either in a north-up or a track-up fashion. 
Previous research has indicated that the use of a north-up as 
opposed to a track-up map yields more accurate map 
reconstructions (a measure of survey knowledge). Also, subjects 
in the map study group were explicitly exhorted to use their full 
20 minutes for studying and envisioning the terrain. 

Following training, all subjects were later challenged to 
fly the exact same route, as prescribed earlier on the map, but 
without the use of the map. This transfer flight was flown on 
the high fidelity E&S visual simulator. Williams and Wickens 
(1993) suggested that such a test without the use of a map might 
provide a "pure assessment" of route and survey knowledge 
promoted by different training exposures of an environment. Only 
a short list of the turn points along the route was available. 
This task of flying the route without the map was assumed to 
provide a measure of subjects' acquired route knowledge. At the 
end of the entire route, subjects were challenged to fly back to 
the original starting point via the most direct flight path (an 
unplanned and unrehearsed event), thus measuring functional 
survey knowledge. After the transfer flight test, subjects were 
asked to reconstruct the map of the entire region solely from 
memory, providing a measure of survey knowledge. 
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Method 

gufe-jg dt s 

Thirty pilots were paid $5.00 per hour for a single three 
hour experimental session. Of the 30 subjects, 26 were males and 
4 were females. Flying experience ranged from 60 to 1500 total 
flight hours with a mean and median of 199 and 111 hours, 
respectively. All subjects had a minimum of a private pilot's 
license with a single engine type rating. These subjects flying 
experience can be further classified as follows: 6 subjects were 
active certified flight instructors, 6 subjects were instrument 
rated pilots all actively pursuing an instructor license, and 18 
subjects were actively pursuing an instrument rating. All 
subjects also had current medical certificates and were screened 
for corrected/uncorrected vision of 20/20 or better. Subjects 
were also current with respect to flying experience (see FAA's 
Federal Aviation Regulation Part 67 for a further explanation). 
Apparatus and Stimuli 

A two axis Flightstick brand joystick was used by subjects 
to control several flight simulations throughout the entire 
experimental session. Subjects manipulated the joystick to 
control the pitch and angle of bank, thereby directly controlling 
altitude and heading respectively. Yaw and thrust were not 
readily controllable but were automatically set by the computer 
simulation. The joystick was mounted to the right arm of the 
subjects' chair. 

The joystick was used in conjunction with 2 different flight 
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simulators. The first simulator used was a Silicon Graphics IRIS 
4D-70GT workstation with a 1280 x 1024 pixel color monitor. The 
monitor's screen measured 34.4 cm horizontally by 27.5 cm 
vertically. When viewed from the seating distance of 110.0 cm, a 
17.4 X 14.0 visual angle (VA) resulted. The minimum screen 
update for simulations on the IRIS was approximately 5 Hz. The 
IRIS depicted three flight variables of attitude, altitude, and 
heading. An attitude indicator was presented in the form of a 
head-up display or HUD. The HUD displayed normal attitude 
symbology, of a fixed aircraft symbol with a rotating artificial 
horizon and pitch ladder, in the center of the screen. An 
altitude indicator was presented vertically along the right 
portion of the display. A moving arrow was used with a fixed 
analog scale, measuring 0 feet on the bottom to 200 feet on the 
top. Altitudes of 140 through 160 feet were highlighted. 
Altitudes above 200 feet were illustrated by the pointer 
disappearing and altitude appearing digitally at the top of the 
scale. A heading indicator was presented digitally, in the 
bottom right portion of the computer screen. All IRIS flight 
simulations approximated a ground speed of 115 knots. In 
addition to the instrument depictions, the IRIS also depicted the 
"visual contact" scene of the world through which the subjects 
flew. 

The other simulator used was an Evans & Sutherland (E&S) SPX 
SOOT image generator and was employed for the transfer flight. 

Two Electrochrome ECP 3000 color projectors were powered by the 
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E&S and were used with two large projection screens, each 
measuring 304.8 x 228.6 cm. Both screens were joined at an angle 
of 115 degrees. One screen was placed directly in front of the 
subject, while the other was joined on the left side (see Figure 
2). This provided subjects with a view spanning from 27 degrees 
right to 85 degrees left of centerline. This gave subjects a 
continuous viewing field of 112 x 38 degrees visual angle when 
viewed from the seating distance of 300 cm. This configuration 
was chosen because most of the required turns in the experiment 
were to the left and also because the simulated aircraft always 
crabbed to the left in the E&S conditions. The screen update for 
the E&S was 50 Hz. All E&S flight simulations approximated a 
ground speed of 85 knots. It is important to note that both 
simulation systems exhibited very similar flight control 
dynamics, in response to stick inputs. 

In addition to differences in visual realism, a second 
difference between the two simulation systems was in the location 
of the flight instruments. As mentioned earlier, the IRIS 
presented the flight instruments on the same monitor that 
displayed the outside visual world. The E&S, however, presented 
only the outside world on the large projection screens. The same 
3 flight instruments mentioned earlier were displayed on a 
separate monitor, directly located 90 cm in front of the 
subjects. This monitor was located beneath the view of the 
projection screens so that it did not block any view of the 
projection screens. This head-down type monitor was the same 
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type used in the IRIS system for the rehearsal flight. 

This experiment exposed subjects to two different geographic 
regions, created specifically for the purposes of this 
experiment. Both simulations systems were capable of presenting 
each of the two geographic regions. The first region was called 
the "warm-up region" and was so named because all subjects flew 
through this region to familiarize them to the control dynamics 
of the flight simulation used throughout the experiment. This 
region was also used to establish a baseline performance measure 
of subjects' simulator control ability. 

The warmup region consisted of a single path appearing on 
the ground which was to be followed in the simulation. The path 
was a continuous course made up of 7 legs (see Figure 3). These 
legs were characterized into 3 categories; 

straight and level (legs 1 and 5) 
straight and hilly (legs 2, 3, and 6) 
curved and level (legs 4 and 7) 

The second geographic area was called the Freemont region, 
and was named so after a small town located in the north-central 
portion of the region. The Freemont region is a square area of 
diverse terrain, measuring approximately 13.5 nautical miles on 
each side and was developed by Williams and Wickens (1993). A 
map of the region was used to initially expose subjects to the 
region (see Figure 4). The map was similar to the VFR 
aeronautical charts commonly used by pilots. The map measured 33 
X 33 cm and was printed to a scale of 1:75618. A specific route 
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was clearly marked on the map as well as many geographic 
landmarks. Topography was indicated by a scale of discrete color 
codes, which made mountain-like formations easily interpretable. 
The topography scale as well as a 1 nautical mile distance scale 
was printed on a section connected to the bottom of the Freemont 
map. When rendered on either the IRIS or the E&S simulation 
system, the features of the world appeared in a three dimensional 
perspective in either an abstract, or highly realistic form, 
respectively. 

Before the experimental procedure began, subjects read and 
signed an informed consent fortn. They also filled out a personal 
data form which asked for their name, age, sex, handedness 
(left/right), FAA flying license/type rating, total flying hours, 
and total cross-country flying hours. 

Subjects were then placed into one of 10 cohorts, based on 
similar flight experience. Within each cohort, subjects were 
assigned to one of 3 experimental conditions known as active 
rehearsal, passive rehearsal, and map study . These conditions 
will be described shortly. 

Next subjects were given a set of instructions. The 
experimenter read these instructions aloud while subjects 
followed along with their own set of instructions. 

Spatial Ability Tests 

Subjects were then given two spatial ability tests called 
the Surface Development and the Cube Comparison tests (Ekstrom, 
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French, Harmon & Derraen, 1976). This session lasted 
approximately 20 minutes. 

IRIS Warm-up Flight 

Next subjects participated in the IRIS warm-up flight 
session which consisted of 3 flights in the warm-up region (see 
Figure 3). Subjects were told of their two goals in this 
session. They were to fly directly over the path depicted on the 
screen and to maintain an altitude of 150 feet above ground level 
(AGL). It was explained that the altitude indicator acted liked 
a radar altimeter, which measured altitude above the ground as 
opposed to above mean sea level (MSL). Subjects were also warned 
that the IRIS was very sensitive to abirupt stick inputs. The 
instructions stressed that the subjects should use small, smooth, 
and precise stick movements to avoid pilot induced oscillations 
(caused by abruptness). 

The first IRIS warm-up flight lasted one minute and it 
exposed subjects to a portion of the warm-up region which was 
characterized as a level path with some turns. This was used to 
demonstrate the control dynamics and the visual flight 
information. The experimenter gave verbal feedback, relating to 
the subjects' initial performance, and reiterated the written 
instruction's advise. 

The second IRIS warm-up flight covered the entire course in 


the warm-up region. This flight lasted approximately 9 minutes. 
At the end of the flight, the monitor presented vertical and 
horizontal RMS errors. This feedback was explained to subjects 
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and it was used as a goal to improve performance. The third IRIS 
warm-up flight was identical to the second except that light 
turbulence was simulated. 

Freemont Training Session 

This was the only session that differed between the 3 
experimental groups. The active group was told that they would 
perform a rehearsal flight in the Freemont region, which would be 
later tested on the E&S system. It was explained that a map of 
the Freemont region would be provided and that a specific route 
was clearly marked on the map (see Figure 4). Their job was to 
fly the most precise route by using both the map and also the 
"outside" landmarks depicted in the simulation. There was no 
path depicted on the ground as in the warm-up region so subjects 
were to use their pilotage navigation skills to complete the 
flight successfully, just as in real cross country flights. In 
addition to course precision, subjects were also told to maintain 
an altitude of 150 feet AGL at all times. It was further 
explained that they would be asked to fly the exact same route on 
a later E&S flight, except that the map of the Freemont region 
and route would not be available. Subjects were strongly urged 
to familiarize themselves with as many geographic landmarks as 
possible. They were told that this strategy could strongly aid 
their success on the E&S flight when they would not have the map 
available . 

The map of the Freemont region and the depicted route were 
then thoroughly explained by the experimenter. While true course 
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headings were printed on the map for each leg, the instructions 
stated that the memorization of such headings would not be a 
guarantee of successful course completion. The possibility of 
simulated winds aloft and turbulence were also mentioned. Again, 
it was stressed that the ability to maintain geographical 
awareness was heavily dependent on the subjects' ability to 
recognize many different landmarks throughout the route and 
region. While successful completion of the route was the primary 
goal, subjects were told to also process the entire region so 
that geographical awareness could be maintained in the event that 
they flew off the route's path. The entire experiment was 
briefly reviewed again. Any questions concerning the map or any 
task were answered. 

The passive group was given very similar instructions except 
it was explained that they were going to view a replay of an 
active subject's rehearsal flight. The map was to be used in 
conjunction with viewing the replay in order to learn essential 
details of the route and region. The ensuing E&S flight without 
the map was fully explained as well. 

After either active or passive subjects were fully briefed, 
subjects were each shown a 2 minute simulation on the IRIS that 
exposed them to many different aspects of the Freemont 
environment. This short simulation showed all of the symbols 
(such as bridges, towers, mountains, and lakes) that might be 
encountered throughout the region on their respective rehearsal 
flights. When the simulation ended, it was explained that it was 
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just an artificial sample of landmarks (arbitrarily placed along 
the flight path) and that it was not part of the Freemont region. 
Active and passive subjects were then given 1 minute to study the 
map before their rehearsal flight began. The rehearsal flights 
lasted approximately 20 minutes. Within each cohort, the 
rehearsal flight path flown by the active subject was digitally 
stored and then replayed for the passive subject. 

Both active and passive subjects were further directed to 
use the map in either a fixed north-up or rotating track -up 
fashion. Cohorts were balanced so that there were an equal 
number of combinations of the 2 factors (rehearsal type/map 
orientation) within each level of relative flight experience. 

Out of the 10 subjects per rehearsal group, 5 subjects were 
instructed to use a track-up map orientation and 5 were to use a 
north-up orientation throughout the rehearsal flight session. 

The third experimental group was the map study group. 

Again, very similar instructions were presented so that this 
group was informed of the same future task of flying without the 
map on the E&S simulator. Through a pure map study session, they 
were to familiarize themselves with the same critical features 
mentioned earlier. A visualization technique was recommended in 
which subjects were told to try to "visualize" flying along each 
leg of the route. The entire route and region was stressed 
again. The map study subjects were given 21 minutes to study the 
map and they were encouraged to devote the full time to active 
study. This 21 minute time was derived in order to equal the 1 
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minute map study and the 20 minute rehearsal flight of the active 
and passive groups. All map study subjects reported that they 
made efficient use of the entire map study session. 

E&S Warm- UP Flight 

After completion of the Freemont training session, all 
subjects were given a 10 minute break and then moved to another 
laboratory for the remainder of the experiment. The identical 
instructions were presented to all three groups for the next two 
sessions, which were flown on the E&S simulator. First, the E&S 
simulator was briefly introduced and the following differences 
between the E&S and IRIS simulations were discussed. The most 
apparent difference was that the E&S used 2 very large projection 
screens. Also, a head-down display was used for the flight 
instruments. The E&S flew at a ground speed of 85 knots, which 
was 30 knots slower than the IRIS simulations. The E&S had a 
much greater graphics capability therefore many features would 
appear more realistic. There was also a simulated haze, so that 
visibility was limited to 5 nautical miles, instead of an 
unlimited visibility as in the IRIS. The E&S would also have a 
direct left crosswind, varying from 6 to 8 knots. This crosswind 
did not come from a fixed world direction (such as from the 
north) but rather it came from a 90 degree angle based on the 
current direction of simulated flight. This created a constant 
crosswind and therefore the flight simulation required a crab in 
order to fly a straight, point to point course. 

The warm-up flight on the E&S was in the same warm-up region 
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as used earlier on the IRIS. The subjects were told that they 
would participate only in one warm-up flight and that this flight 
would be used for the same purposes of familiarization as on 
earlier flights. After the warm-up flight ended, feedback was 
again given. Performance on this flight was used to establish if 
all subjects entered the following transfer flight with 
approximately equivalent simulator control abilities. The E&S 
flight through the warm-up region lasted approximately 9 minutes. 
E&S Transfer Flight 

Instructions were given to brief all subjects on the final 
flight, the E&S evaluation (transfer) flight. The subjects were 
reminded that they would be challenged to fly the exact route 
depicted earlier, but without the aid of the Freemont map. 
However, a reference card listing all of the 7 turn points on the 
route was provided for the subjects to use throughout the 
evaluation flight (see Figure 5). These 7 points were also 
briefly reviewed. A few guidelines to follow were also 
explained. For instance, if subjects had realized that they flew 
off the desired course they were to do their best to reintercept 
the exact course depicted earlier and not to simply fly a direct 
route to the following turn point. Subjects were also briefed on 
the possibility of getting geographically disoriented or lost. 
This was discussed simply as a possibility and it was not 
represented to them as a "bad" event. If subjects were certain 
that they had become lost, they were to immediately inform the 
experimenter. In this case, the experimenter would then issue 
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vectoring instructions to steer subjects back to the desired 
route. The subjects were told to verbally declare when they were 
able to continue on the route. If they continued to remain 
disoriented, the experimenter would continue to give directions 
to the subject until they were able to continue. Unknowii'^q to 
the subject, the experimenter would record all pertinent 
ii formation such as location, headings, length of disorientation, 
etc., which would be used for later analysis. 

When subjects had nearly completed the entire route (i.e., 
nearing the end oh leg 7 in Figure 4), they were told to perform 
a unexpected task or flying back to the initial point (IP). Once 
subjects were approximately 30 seconds from the final point (the 
lake), they were told to visualize the entire map of the region, 
where the final point (the lake) was located and where the 
initial point (IP) was located. In addition, subjects were told 
to do their best to quickly establish the correct relationship 
between the lake and the IP. Once subjects actually arrived at 
the lake, they were instructed to fly as directly as possible to 
the IP. Subjects were reminded, as they were flying, to use 
their heading indicator to aid them in flying in a particular 
direction. Any maneuvers, such as S turns and 360 degree turns, 
were allowed if deemed necessary. Finally, subjects were told 
that they had to navigate within a 1/4 nautical mile radius of 
the IP in order for the simulation to automatically stop. 

Subjects were allowed to end the simulation if they felt that 
they were not able to successfully arrive at the IP. The 
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experimenter was also able to end the simulation and did so if a 
subject was about to fly off the edge of the simulated region. 

The total time to fly the route in the transfer task and the 
attempt to the IP lasted approximately 35 minutes. 

Immediately after the simulation, subjects were moved to a 
desk and were told to reconstruct the map of the Freemont region. 
The reference card of the turnpoints was not available for use. 

A standard sheet of white paper, a pencil, and a large eraser 
were provided. The top of the paper had a small margin for the 
subjects name, while the bottom portion was sectioned off by a 
black line so as to form an 8 1/2 inch square area. Since the 
map of the Freemont region existed as a square, the same 
proportional area was given to the subjects. Subjects were asked 
to replicate every single detail from the Freemont map, including 
both the route and terrain features. Before subjects were 
allowed to start, they were informed of a potentially useful 
strategy to use while drawing the region. This strategy involved 
first sketching out the route on the back side of the paper. 

This would allow subjects to visually see if their first drawing 
of the route matched their memory of how the route actually 
existed. If they interpreted their route as being skewed or out 
of scale, subjects would then have the opportunity to make the 
necessary corrections on the front side of the paper. After the 
route was proper placed, the remainder of the landmarks could 
then be placed throughout the region. This strategy was utilized 
by 21 of the 30 subjects and it was reported to be very helpful. 
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The use of an eraser was highly encouraged as well, so that the 
representation of subjects' memory could be accurately recorded. 
Subjects were given 10 minutes for this task. 


After the map reconstruction task was completed, subjects 
were then fully debriefed on their performance throughout the 
entire experiment. 

Dependent Measures 

Vertical and horizontal deviations scores were recorded for 
all flights in both the warm-up and Freemont regions. These 
error scores were in the form of root mean square error (RMSE) 
and mean absolute error (MAE). The horizontal or tracking error 
was measured from the ideal ground track while the vertical error 
was measured from the ideal altitude of 150 feet AGL. Both types 
of error scores were recorded for vertical and horizontal 
deviations and this data provided evidence for subjects' route 
knowledge. 

The RMSE method has been traditionally used to measure error 
dispersion in tracking tasks (Kelly, 1969). However, the 
statistical assumptions (of a normal distribution with a mean of 
zero) of the RMSE method may not be suitable for the horizontal 
tracking scores in this experiment. This is because subjects 
tended to err generally to one side of any particular leg. In 
other words, subjects' flight path error was characteristic of 
being to one side of the desire route and not crossing back and 
forth over the ideal ground track. RMSE calculations also tend 
to inflate large error scores, such as the horizontal scores in 
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this experiment. Given these circumstances, the MAE method was 
used to statistically analyze the horizontal error scores while 
the RMSE method was used for the vertical scores. These choices 
were consistent with the methods used by Williams and Wickens 
(1993) . 

Subjects' performance on the surprise challenge of flying 
back to the IP was recorded by a computer plot of subjects' 
actual flight path, relative to the lake and the IP (see Figure 6 
for an example). The initial heading errors flown by subjects 
were figured by using a protractor to measure the angular 
disparity between subjects' initial heading and the correct 
heading and this data was used to provide evidence for subjects' 
functional survey knowledge. 

Finally, subjects' survey knowledge was measured from their 
hand drawn depictions of the Freemont map. These maps were 
scored by several methods. Given that the hand drawn maps were 
somewhat distorted and not ideally constructed, two prominent 
features were identified to exist in all of the hand drawn maps. 
These features were identified as (1) the series of roads/oil 
storage tanks (actually existing in the north-central portion of 
the region) and (2) the saddle-shaped ridge formed between the 
two large mountains (actually existing in the south-central 
portion of the region). Both of these prominent features also 
had the route depicted through them. A transparent template 
having an equivalently proportioned route and selected landmarks 
was then used to match the two prominent features previously 
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described. These two features on the hand drawn maps were 
aligned with their corresponding counterparts on the template and 
a newly defined north-south axis was defined. The angular 
disparity between the new (distorted) north-south axis and the 
actual north-south axis was measured. Furthermore, the template 
was accordingly fastened to the hand drawn maps in order to 
determine other measurements. The other measurements consisted 
of scores called Landmark, Placement, and Angle. The Landmark 
score was computed by the number of preselected landmarks 
actually drawn by a subject. A Placement score was calculated by 
the sum of the difference in distances between where landmarks 
were drawn and where their corresponding representation was 
located on the template. An Angle score was calculated by 
measuring the absolute angular disparity between where each of 
the 7 legs existed on the template and how these legs were drawn 
on the maps. 
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Results 

During the experiment, there were certain circumstances that 
required the replacement of four subjects, and therefore a total 
of 34 subjects actually participated in the experimental process. 
These four subjects were replaced for the following reasons. 
First, equipment failures in the EtS system did not allow two 
subjects to complete the entire experiment. Secondly, two other 
subjects did not meet minimum pre-transfer flight criteria 
relating to simulator control and ability to navigate on the 
rehearsal flight. These two subjects were not able to 
successfully "control" the simulator on the IRIS warm-up flights. 
(Control was defined as being able to follow the depicted line in 
the warm-up region and not become totally disoriented from 
following the line.) Furthermore, the substandard simulator 
control abilities of these subjects influenced their performance 
on the following IRIS rehearsal flight so that these two subjects 
were not able to navigate on the IRIS rehearsal flight. 

All results in the following analyses will be reported as 
significant, if in fact p values are less than .05. However, all 
E values less than or equal to .15 will be reported to insure 
that any "marginally" significant results effects are brought to 
the reader's attention. 

Spatial Ability Tests 

The scores for each of the two spatial ability tests were 
subjected to one-way ANOVA's to determine if any differences 
existed between the three training groups. The analysis of each 







test did not reveal any significant difference between the 
training groups. 
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E&S Warm-up Flight 

In order to establish whether all groups entered the 
transfer flight with equal levels of simulator control ability, a 
two-way fixed effects ANOVA (3 levels of training group by 7 
levels of leg) was performed on each of the horizontal and 
vertical error data sets (SAS Institute Inc., 1985). Both the 
horizontal MAE and vertical RMSE data used in the following 
analyses were transformed by a logarithmic based 10 function, in 
order to meet assumptions of normality. For the horizontal MAE, 
there was a significant main effect for leg (F(6, 189) = 11.3, p 
< .001), no main effect for training group, and no leg by 
training group interaction. The effect for leg was expected and 
similar to that observed by Williams and Wiclcens (1993) since the 
highest horizontal MAE occurred on the curved legs, which are 
more difficult to follow than the straight legs (see legs 4 and 7 
on Figure 3). 

The ANOVA performed on the vertical RMSE data showed a 
significant main effect for leg (F(6, 184) = 33.0, p < .001), no 
main effect for training group, and no leg by group interaction. 
The effect for leg was expected since the highest vertical RMSE 
occurred on the hilly legs (see legs 2, 3, and 6 on Figure 3). 

Due to a minor malfunction with the E&S system, some vertical 
error data for one subject (five of seven legs) was not included 
in the vertical RMSE ANOVA and was treated as missing values in 
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the analysis. 

The absence of significant effects or interactions involving 
training group, up to this point, indicates that all three 
training groups entered the transfer flight with essentially 
equal spatial ability and equal ability to control the E&S 
simulator. Hence subsequent differences can be more directly 
attributed to differences in training conditions. 

&S- .Iraasfgr. g l ig ht 

Out of the 30 subjects participating in the transfer flight, 
there were 8 times when a subject declared being lost or 
disoriented. These incidents occurred in each training group as 
follows; 4 active svibjects, 3 passive subjects, and 1 map study 
subject. No subject was disoriented more than one time per 
flight. Of the seven subjects in the active and passive 
rehearsal training groups who became disoriented, six of those 
subjects had been trained in the north-up map condition. 

In order to meet assumptions of normality, both the 
horizontal MAE and vertical RMSE data were transformed by a 
logarithmic based 10 function. Next, the horizontal MAE data 
were subjected to a two-way fixed effects ANOVA (3 levels of 
training group by 7 levels of leg) in order to assess how 
accurately the three training groups navigated along the route in 
the Freemont region. A graph of the group by leg horizontal MAE 
means is depicted in Figure 7. Both main effects of training 
group (F(2, 189) = 4.6, E = .012) and leg (F(6, 189) = 3.9, £ < 
.001) were found to be significant. Additionally, there was a 
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marginally significant interaction between training group and leg 
(£(12, 189) = 1.6, E = .090)• The source of this interaction 
appears to be the generally greater variability of performance 
across legs experienced by the map study group, in contrast to 
the two rehearsal flight groups. 

In order to determine the extent to which differences 
between pairs of training groups contributed to the overall 
effect on horizontal MAE, three separate two-way fixed effects 
ANOVAs (2 levels of training group by 7 levels of leg) were 
performed on the horizontal MAE data. These ANOVAs revealed that 
the map study group had significantly less horizontal MAE than 
the active group (£(1, 126) = 8.3, e < .005) and that the passive 
group had less horizontal MAE than the active group at a 
marginally significant level (£(1, 126) = 3.4, e < *069). (It 
should be noted that these tests were post hoc. If a more 
conservative adjustment of significance level were undertaken, 
the latter effect would be considered insignificant and hence it 
might be concluded that there were no differences in learning 
between passive and active rehearsal flights.) The two level 
ANOVA revealed no difference between map study and passive 
groups. 

A two-way *!ixed effects ANOVA carried out on the vertical 
RMSE data did not reveal any significant main effect or 
interaction involving training group. However it did reveal a 
significant main effect for leg (£(6, 189) = 21.4, e < -001). 

This effect was predicted from the warm-up flight and was 










consistent with the results in Williams and Wickens' (1993) 
experiment. The legs which included mountains suffered the 
greatest vertical RMSE. 
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Since the independent variable of map orientation (north-up 
versus track-up) was only manipulated among the two rehearsal 
flight groups (active and passive), a three-way fixed effects 
ANOVA (2 levels of training group by 2 levels of orientation by 7 
levels of leg) was performed on the horizontal MAE data for these 
two groups. This analysis revealed a significant interaction of 
training group by orientation (£(1, 112) = 4.0, p = .048) as well 
as a marginally significant main effect of group (F(l, 112) = 

3.4, E = .068). This interaction is depicted in Figure 8, and 
reveals that the cost in lateral tracking performance for the 
active training group was experienced exclusively by those 
subjects who were requested to fly the rehearsal flights with 
their map held in a fixed, north-up orientation. 

Last Lea Performance 

Subjects' performance on flying the "surprise" last leg from 
the lake to the initial start point was analyzed from the data of 
the last leg plots. A typical plot of a subject's track 
following this request is shown in Figure 6. From these data 
plots, the absolute values of the angular disparities between 
subjects' initial headings and the ideal heading were calculated 
for each subject and these deviations were subjected to a one-way 
ANOVA. This analysis did not reveal a significant effect for 
type of training group. Next, a correlation was calculated 
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between how subjects drew the last leg relationship on the hand 
drawn maps and the initial direction in which subjects actually 
flew the last leg (i.e., the vector calculated in Figure 6). 

This calculation resulted in a nonsignificant correlation 
coefficient of r = .13. 

Map Reconstruction 

The hand drawn map reconstructions were graded on three 
separate scores of Landmark, Placement, and Angle. A single one¬ 
way ANOVA (3 levels of training group) was performed on each of 
the three map reconstruction scores. The first analysis was 
based upon the number of significant landmarks correctly placed 
which subjects in the rehearsal flight group could have seen 
within their forward field of view while navigating along the 
desired route. This analysis revealed a significant main effect 
of training group for Landmark score (F(2, 27) = 6.46, p < .006). 
In order to determine which groups differed significantly, a 
Bonferroni multiple comparison t-test was performed on the 
training group means. This test revealed that the map study 
group had a significantly larger Landmark score average of 18.0 
than the active group average of 13.7 {t(.99, 25) = 2.57, p < 
.05). The average score of the passive group (15.9) fell in 
between the two other scores and did not significantly differ 
from either. 

In order to assess the accuracy of placement of the 
landmarks that were drawn, it was first necessary to "orient" the 
subject drawn map to the frame of reference of the true map. 
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This was done by aligning the primary north-south axes of the 
maps. In order to make such an alignment, two prominent north 
and south features, universally common to all subject drawn maps, 
were identified to exist. These features were the vertical 
stretch of roads/oil tanks which intersects the route in the 
north central portion of the region, and the mountain pass 
crossing which the route traverses on the first leg in the south 
central region (see Figure 4). This alignment was accomplished 
by rotating a subject drawn map so that these two landmarks were 
aligned with the corresponding landmarks on the map template. 
Furthermore, the placement of the subject map along this defined 
axis was fixed so that these two prominent intersections were 
placed equidistantly apart from their counterparts on the map 
template. Once this alignment was established, the absolute 
distances between the drawn and true landmarks were calculated 
from the critical landmarks identified in the first analysis. 

The mean value of these distances was then computed for each 
subject, as the Placement score. 

The second ANOVA revealed that the training groups differed 
significantly with respect to Placement score {F(2, 25) = 5.0, p 
= .015). A Bonferroni t-test indicated that the passive group 
Placement score error of 15.3 was significantly less than the 
active group error of 21.8 (t(.99, 25) = 2.57, p < .05). Also, a 
marginally significant difference existed between the map study 
group Placement score average of 17.2 and the active group 
average of 21.8 {t(.98, 25), p < .10). 










The third ANOVA, testing the Angle score, did not reveal 


significant differences between the training groups. 
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Discussion 

The main purpose of this experiment was to evaluate the type 
and amount of geographic knowledge promoted by different training 
methods. The performance of groups receiving the three different 
training methods (map study, active rehearsal, and passive 
rehearsal flight) was measured on several transfer tasks, that 
were designed to assess route knowledge, survey knowledge, and 
the use of survey knowledge to solve functional navigation 
problems. 

Pre-Transfer Flight Performance Measures 

It was important to verify that the three training groups 
entered the transfer portion of the experiment with equal spatial 
ability and equal simulator control ability. These two 
assumptions were validated by the findings that there were no 
significant differences between training groups on either the two 
spatial ability tests or on the E&S warm-up flight. 

Route Knowledge 

The E&S transfer flight was used to assess the "pure" 
transfer of route knowledge which was promoted by each of the 
three training methods. The results revealed that the map study 
group performed significantly better than the active group and 
performed as well as the passive group, thereby reaffirming the 
real benefits of map study (specifically found in the Williams 
and Wickens, 1993, study). In fact, the benefits of map study 
compared to the other methods may have even been enhanced in the 
present experiment, relative to the findings of Williams and 
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Wickens. In their study the map training condition was no better 
than the active training condition in supporting route knowledge, 
whereas in the present study it was superior. 

Two reasons may be offered for the improvement in map study 
performance observed here. First, the methodology employed in 
the current experiment allowed for a "pure assessment" of route 
knowledge in the transfer flight. Map access was not available 
during the transfer flight, whereas all training groups in the 
Williams and Wickens study had such access. Therefore those 
rehearsal flight training groups did not have to rely solely on 
their memory (geographic knowledge) in order to successfully 
navigate along the route. Those training groups were also aware 
of the map's future availability therefore they knew that they 
could simply reference the map throughout the transfer flight. 

The training groups in the current study, however, did not have 
any outside reference in order to confirm position on the route 
and within the region on the transfer flight, therefore these 
groups had no other option but to adequately learn the route 
during the training session. Without a perceptual representation 
of the region to refer to on occasion, their performance may have 
suffered accordingly. 

A second explanation for the enhancement of the map study 
group's performance is based upon the increased motivational 
measures used to exhort subjects' effort to learn the route and 
region (as compared to the Williams and Wickens, 1993, study). 

In the training session of the present experiment, al"' three 
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groups were repeatedly reminded of the absence of the map in the 
forthcoming transfer flight. These multiple "warnings" of the 
map's absence were specifically used to maximize the learning of 
the critical geographic aspects necessary for successfully 
navigating on the transfer flight, and in particular to encourage 
all subjects in the map study condition to take full advantage of 
the study time available to them. Corresponding instructions 
were not issued to the map study group in the Williams and 
Wickens study leading, perhaps to a selectively greater 
improvement in knowledge for this group in the current study. 

The finding that the active group navigated more poorly than 
the other groups was surprising since Williams and Wickens (1993) 
found that the active group performed as well as or better than 
the two other groups. Two possible explanations are offered that 
may account for this reversal of findings between the two 
studies. First, the increased workload of a shorter flight 
duration (20 minutes as compared to 30 minutes to cover the same 
distance) may have hurt the active group more than the passive 
group. The task of navigating the route (with reference to the 
map) while actively flying at a faster speed and trying to hold a 
particular altitude may have exhausted valuable spatial resources 
of the active group, which are necessary to encode the 
geographical cues of route knowledge into memory. The passive 
group did not have to contend with this complex task and 
therefore could devote their full attention to learning the 
route. Williams and Wickens speculated that the added effort of 
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active involvement in the environment (during their training 
session) required the investment of more effort by the active 
subjects than was required of their passive counteirparts. 
Therefore such added effort was inferred to result in a more 
accurate mental representation of the region for the active 
group. However, the faster training flight (resulting in a 
shorter training session) in the present experiment may have 
overtaxed the spatial resources of the active group to the extent 
that their acquisition of route Icnowledge suffered. We speculate 
here that the detrimental effect of a shorter time (and therefore 
a faster flight) would only exert itself as a worlcload stressor 
on those subjects who were actually in the control loop, actively 
flying (i.e., the active group; not the passive nor map study 
group). The reduced "study time" for these latter two groups 
would have a less disruptive effect on learning, since it was not 
accompanied by the higher worlcload of a higher bandwidth flying 
task. 

The second explanation as to why the active group performed 
more poorly in the present study is based on the fact that half 
of the subjects in the active group also had to mentally rotate 
the information from a fixed, north-up map while actively flying 
the rehearsal flight simulator. Figure 8 reveals that it was 
only this subset of the active subjects that was responsible for 
the poorer performance of the active group. The added spatial 
cognitive demands of mental rotation (Aretz, 1991) are assumed 
here to compete, both with the spatial demands of controlling the 
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simulation flight and of storing in memory, the surrounding 
navigational features. This argument is further supported by the 
extent to which the rehearsal flight groups using a north-up map 
became lost during the transfer flight. Four of the five active 
training subjects who became lost, were using north-up maps 
during the training flight. 

The analysis of the vertical RMSE data revealed no 
significant differences between the three training groups, 
indicating that there were no tradeoffs between horizontal 
tracking performance and vertical control accuracy. If, in fact 
the spatial resources of the active training groups were 
overburdened during the rehearsal training session (leading to a 
poorer acquisition of route knowledge), their subsequent altitude 
control performance on the transfer flight was not significantly 
affected. 

In conclusion, the advantages of actively being "in the 
loop" may have not only been neutralized, but may have actually 
been reversed, by the excessive workload demands of controlling a 
fairly difficult rehearsal flight. 

Survey Knowledge 

As Thorndyke (1980) has described, survey knowledge is a 
global, map-like representation of the geographic properties of 
an environment. It is then reasonable to assume that people with 
the strongest survey knowledge should be able to accurately 
illustrate this knowledge by drawing a map of a particular 


environment. 
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Three scores were computed to compare the accuracy of 
subject drawn maps, and these data showed that the active group 
performed significant worse than either the map study group or 
the passive group, with respect to the Landmark and the Placement 
score. These results are only partially congruent with the 
findings in the Williams and Wickens (1993) study. In agreement 
with the current study, they found the map study group had better 
survey knowledge than the active group. However in contrast to 
the current study, Williams and Wickens found the passive group 
to show survey knowledge that was equivalent to that of the 
active subjects, and not to the map study subjects. Why the 
passive group subjects improved their performance here, compared 
to the Williams and Wickens study, is not immediately apparent. 

The other significant difference in survey knowledge was 
found in the Placement score, which resulted in the active group 
exhibiting significantly larger landmark placement errors than 
the passive group. These results were again inconsistent with 
the findings of Williams and Wickens (1993) since they did not 
report any significant differences between these two training 
groups. The fact that the active group in the current study, 
again performed significantly worse on a measure of survey 
knowledge than one of the other groups may be attributed to the 
same phenomenon of spatial resource competition discussed 
previously. The map reconstruction data from Aretz (1991) 
supported the existence of a spatial resource competition between 
the acquisition of survey knowledge navigation and flight control 
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demands. Such competition was used to explain his finding that a 
passive (autopilot) group correctly placed more landmarks on a 
recalled map than an active (manual control) group. 

If, in fact, the active group in the current study was 
spatially overburdened during the rehearsal training session, 
they may not have been able to develop a strong survey knowledge 
of the environment. Their lack of survey knowledge may have been 
further hindered by their subsequent poor performance on the 
transfer flight (i.e., for those trained with a north-up map, 
getting lost more than other groups and poorest performance on 
navigating the route), It is reasonable to assume that the 
development of survey knowledge by the active group may have been 
disrupted by their attempts to keep oriented and navigate along 
the route in both the training condition and in the transfer 
flight. These difficulties would be far less pronounced for the 
other two groups, who did not encounter the competing demands of 
flying during training. 

Williams and Wickens (1993) did draw the general conclusion 
that the map study group had acquired a stronger survey knowledge 
than both of the rehearsal flight groups. The results of 
Thorndyke and Hayes-Roth (1982) also supported that conclusion 
based upon the fact that the low time navigation groups were 
outperformed by the map study groups in survey knowledge 
judgments of straight line distance and object location. In the 
current study, while the map study group also performed better 
than both rehearsal flight groups in Landmark recall, the 
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superiority over the passive group was not statistically 

1 

i significant. For the Placement error measure, the passive group 

actually performed slightly better. The reason for the overall 
improvement in survey knowledge for the passive training group, 
is not immediately apparent. 

Functional Survey Knowledge 

Williams and Wickens (1993) tested subjects' ability to 
apply their survey knowledge for the purposes of navigating on an 
unanticipated task. Their results were interesting in that the 
conditions (i.e., map study) that promoted good survey knowledge 

i 

(map drawing) did not appear to be particularly effective in 
promoting the functional use of this knowledge. Furthermore, 
they found that neither of their different groups were very 

I 

successful on this sort of navigational task. The present study 
! tested subjects' performance on the same task (returning directly 

to the starting point of the flight), and measures of initial 
headings were compared between the three training groups. 
Consistent with the Williams and Wickens study were the findings 
j that the three training groups did not significantly differ on 

I 

this task, despite their differences in performance on the 

I 

j subsequent map drawing task. Additionally, the correlation 

between subjects' initial headings and their hand-drawn bearing 
relations to the initial point did not indicate a significant 
' relationship. For example, if subjects depicted (from their map 

drawings) the lake-IP relationship with a correct bearing (080 
degrees) or even an incorrect bearing (100 degrees), there was no 
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evidence that subjects applied this correct (or incorrect) survey 
knowledge with any corresponding equivalency to the initial 
portion of the flying task. These results suggest that what may 
be considered subjects' "inferred" survey knowledge on the basis 
of their reconstructed map features, may be difficult to access 
and use in the solution of a navigation flying task. 

Conclusions 

The key finding in this study is that the active training 
group consistently performed more poorly than, or at least no 
better than either of the two other groups for every dependent 
measure. This finding generali^es the conclusion that the active 
training group acquired the weakest versions of route, survey, 
and possibly functional survey knowledge. Conversely, the value 
of map study was highlighted by poorer performance of the two 
rehearsal groups on certain measures of geographic knowledge 
assessment. Another interesting finding was that the passive 
group tended to fall in between the other two groups on many of 
these tasks. The potential benefits from "seeing" the three- 
dimensional aspects of a future flight's environment did not seem 
to outweigh the performance based on knowledge acquired from a 
two-dimensional map study session, particularly when this 
"seeing" process was coupled with continuous active flight 
control, and resource demanding mental map rotation. 

If, indeed, spatial resources are in high demand for the 
rehearsal flight groups (specifically the active training group) 
then the question might be raised of how to properly alleviate 
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this potential problem. One answer to this may be to simply 
decrease spatial workload during a rehearsal flight session. In 
terms of the present study, this might be accomplished by several 
methods. First, the flight speed can be decreased (as it was in 
the Williams and Wickens, 1993, study) such that rehearsal groups 
are less burdened when learning a particular route. Slowing down 
the speed for the active rehearsal flight group may allow more 
resources to be diverted from flying to learning of the route and 
region. A somewhat related option of reducing time was 
highlighted in the Martin and Lidderdale (1992) study where 
rehearsal flight pilots had the option to "freeze" and "slew" in 
the simulation. 

A second remediation may be to reduce the demands of flight 
control skills of the active rehearsal flight group. For 
example, eliminating the need for altitude control will allow the 
active group to only have to make horizontal tracking inputs, 
thereby reducing the competition between spatial demands for 
flying, navigating, and route learning. A further reduction of 
flight control demands may be accomplished by having an 
"autopilot" option, which keeps the simulation flying along a 
programmed heading with active learner inputs only occurring at 
key "checkpoints" on the route. If the harmful effects of 
limited spatial demands in the active condition can be 
circumvented, then the potential benefits of "seeing" the 
environment and "actively" making navigational decisions may 
outweigh the benefits of other methods of passive viewing or map 
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Study preparation. 

For applied purposes, these three training methods should 
not be considered to be an "all or none” decision for the uses of 
flight preparation. Although lacking experimental control, the 
U.S. Air Force studies (Martin & Lidderdale, 1983; Nullimeyer, 
Bruce, Conquest, & Reed, 1992) mentioned earlier, indicated that 
active rehearsal flights may be a beneficial addition to a 
thorough map study session. The current results have called 
attention to the critical role that navigational workload may 
play in inhibiting or neutralizing the value of those rehearsal 
flights. 

The results of this study also have real theoretical 
implications for the understanding of how people, not only 
acquire geographic knowledge, but how they act upon such 
knowledge. The discrepancies between subjects' reported survey 
knowledge (through hand drawn maps) and how they used this 
knowledge can perhaps be explained by the spatial resource 
competition described earlier. In order to gain a more in-depth 
understanding of this apparent production problem, a future 
experiment should be designed such that the spatial demands are 
systematically varied to determine their influences. 

However, the current methods in which survey knowledge is 
obtained (i.e., through hand drawn maps) may need to be re¬ 
evaluated or possibly replaced. Distortions existing in hand 
drawn maps from their true states have been clearly documented 
(Battiste Sc Delzell, 1991; Chase & Chi, 1979; Stevens & Coupe, 
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1978; Tversky, 1991). Maybe such a method is not appropriate for 
the measurement of survey knowledge. Also, there may exist 
significant differences between individual map drawing ability 
skills (i.e., manual dexterity differences) which can influence 
the "score" a hand drawn map would receive. Differences in 
individual renderings of a geographic environment may only be 
filtered out by the recruitment of large subject pools. If such 
distortions in hand drawn map can be clearly accounted for and 
possibly transposed, hand drawn maps may then be an efficient 
method of measuring one's survey knowledge. 

































Ficfure 3 . Map of the Warm-up region and route used on both 
simulators. 
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Figure 4 . Map of the Freemont region and route. 
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PREEMQNT ROUTE TURN POINTS 

PROM INITIAL POINT (IP) TO: 

1. AIRSTRIP 

2. RIVER INTERSECTION 

3. BRIDGE 

4. LAKE 

5. OIL TANKS 

6. AIRSTRIPS 

7. LAKE 


Figure 5 . Reference card of the route's turnpoints available 
during the E&S transfer flight. 












SUBJECT 83 


64 



Figure 6 . An example of a plot used to calculate the initial 
heading flown on a subject's last leg. 










Figure 7 . Mean log horizontal MAE as a function of leg for the 
three training groups on the transfer flight. 















ACTIVE PASSIVE 

REHEARSAL FLIGHT GROUP 


Ficmre 8 . Mean log horizontal MAE as a function of map orientation 
used by the two rehearsal flight training groups on the transfer 
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